Noise tolerant algorithms for learning and searching

We consider the problem of developing robust algorithms which cope with noisy data. In the Probably Approximately Correct model of machine learning, we develop a general technique which allows nearly all PAC learning algorithms to be converted into highly e cient PAC learning algorithms which tolerate noise. In the eld of combinatorial algorithms, we develop techniques for constructing search algorithms which tolerate linearly bounded errors and probabilistic errors. In the eld of machine learning, we derive general bounds on the complexity of learning in the recently introduced Statistical Query model and in the PAC model with noise. We do so by considering the problem of improving the accuracy of learning algorithms. In particular, we study the problem of \boosting" the accuracy of \weak" learning algorithms which fall within the Statistical Query model, and we show that it is possible to improve the accuracy of such learning algorithms to any arbitrary accuracy. We derive a number of interesting consequences from this result, and in particular, we show that nearly all PAC learning algorithms can be converted into highly e cient PAC learning algorithms which tolerate classi cation noise and malicious errors. We also investigate the longstanding problem of searching in the presence of errors. We consider the problem of determining an unknown quantity x by asking \yes-no" questions, where some of the answers may be erroneous. We focus on two di erent models of error: the linearly bounded model, where for some known constant r < 1=2, each initial sequence of i answers is guaranteed to have no more than ri errors, and the probabilistic model, where errors occur randomly and independently with probability p < 1=2. We develop highly e cient algorithms for searching in the presence of linearly bounded errors, and we further show that searching in the presence of probabilistic errors can be e ciently reduced to searching in the presence of linearly bounded errors. Thesis Supervisor: Ronald L. Rivest Title: Professor of Computer Science